Multimodal Assessment of Conversational Engagement in Persons with Parkinson’s Disease

Our capacity to engage in meaningful conversations depends on a multitude of communication cues: verbal delivery of articulate and intelligible speech, tone and modulation of voice, exhibition of a range of facial expressions, and display of body gestures, among others.Parkinson’s disease diminishes verbal and non-verbal communication facilities in affected persons. Occupational therapists measure the expressive behavior of persons with Parkinson’s through observation and interaction. In this paper, we propose a computational model of measuring expressiveness and engagement. We record persons engaged in conversation and compute descriptors from multiple modalities (face, body, voice). We train a regression model from these multimodal features to scores obtained from domain experts and seek to answer the following questions. First, can a machine learning model be trained to accurately score expressive behavior? Second, what multimodal features are important in order to assess the level of conversational engagement? Third, what is the minimum length of observation required by the model to make an accurate prediction?